Adjudicating threat levels in computed tomography (ct) scan images

ABSTRACT

An improvement to automatic classifying of threat level of objects in CT scan images of container content, methods include automatic identification of non-classifiable threat level object images, and displaying on a display of an operator a de-cluttered image, to improve operator efficiency. The de-cluttered image includes, as subject images, the non-classifiable threat level object images. Improvement to resolution of non-classifiable threat objects includes computer-directed prompts for the operator to enter information regarding the subject image and, based on same, identifying the object type. Improvement to automatic classifying of threat levels includes incremental updating the classifying, using the determined object type and the threat level of the object type.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/735,684, filed May 3, 2022, entitled “Digital Unpacking of CTImagery,” which is a divisional of U.S. patent application Ser. No.17/499,537, filed Oct. 12, 2021, entitled “Digital Unpacking of CTImagery,” which claims the benefit of U.S. Provisional PatentApplication No. 63/160,450, filed Mar. 12, 2021, entitled “DigitalUnpacking of CT Imagery,” all of which are hereby incorporated byreference in their entireties.

STATEMENT OF GOVERNMENT INTEREST

The present invention was made by employees of the United StatesDepartment of Homeland Security in the performance of their officialduties. The U.S. Government has certain rights in this invention.

FIELD

Embodiments disclosed herein generally relate to electromagnetic (EM)scanning.

BACKGROUND

In various systems for electromagnetic (EM) scanning of containers forpotential threat items, e.g., airport EM baggage scanning systems, an EMscanning equipment sends an operator a visible image of the EM scan ofthe entire container, accompanied by a request to the operator toidentify which, if any, among the content items requires further action,e.g., manual inspection. In various of such systems, the operator drawsa bounding box around each content item requiring further attention.Baggage items, though, can contain visually obstructive distributions,e.g., stacking and bunching, of many different kinds of items. This canmake the operator task of mentally separating the item images from oneanother and determining, for each, whether it's an image of a threatitem, a benign item, or an unknown item, time and labor intensive.Further, upon operator reception of a stream of baggage items,cumulative time and labor of separating and adjudicating their contentimages can result in operator errors.

SUMMARY

Systems are disclosed and an example can include a computer-based systemfor adjudicating object images in a computed tomography (CT) scan imagesof item containers, including a processor, and a tangible, non-volatilememory coupled to the processor and storing processor-executableinstructions for the processor to: receive an image data, the image databeing a CT scan of the container item, perform a separation process onthe image data to obtain object images, perform a classification processon the object images between being in a threat item image class, abenign item image class, and a non-adjudicable threat image class; andcommunicate to a graphical user interface (GUI), as subject images,object images classified as non-adjudicable image class, the GUI beingassociated with an operator workstation. The processor executableinstructions include instruction for the processor to generate anddisplay on the GUI prompts for operator inputs regarding the subjectimages, and based at least in part on the operator inputs, identify theobject type of the subject image, and based at least in part on theobject type of the subject image, identify a threat level of the subjectimage, and update the classification process, based at least in part onthe object type of the subject image and the threat level of the objectimage.

Methods are disclosed and an example can include receiving a CT scanimage of an item container, including object images, classifying theobject images between adjudicable threat level object images and unknownthreat level object images. The example can include, responsive to aresult of the classifying including an unknown threat level objectimage, displaying on a display of a graphical user interface (GUI) of anoperator, as a subject image, an unknown threat level object image,together with a prompt for the operator to enter, via the GUI, aparticular information regarding the subject image. The example caninclude based at least in part on the operator entered information,identifying the object type of the subject image, and based at least inpart on the object type of the subject image, identifying the threatlevel of the subject image. The example can include updating theclassifying, based at least in part on the object type of the subjectimage and the threat level of the object image.

Other methods are disclosed and an example can include receiving a CTscan image of an item container, digitally unpacking the object images,to obtain a plurality of unpacked object images, classifying by a firstthreat level classifier the unpacked object images, results from thefirst threat level classifier including first classifier known clear,first classifier known alarm, first classifier known clearable, firstclassifier known suspect, and first classifier unclassifiable. Theexample can further include classifying by a second threat levelclassifier the unpacked object images, results from the second threatlevel classifier including second classifier known clear, secondclassifier known alarm, second classifier known clearable, secondclassifier known suspect, and second classifier unclassifiable,conferring between the results from the first level threat levelclassifier and the second level threat level classifier, based on aresult of the conferring, re-assigning the threat levels.

Other features and aspects of various embodiments will become apparentto those of ordinary skill in the art from the following detaileddescription which discloses, in conjunction with the accompanyingdrawings, examples that explain features in accordance with embodiments.This summary is not intended to identify key or essential features, noris it intended to limit the scope of the invention, which is definedsolely by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example environment for systems and methods forcomputer tomography (CT) scanning of containers, providing digitalunpacking, object threat level classification, computer directed queryto resolve unknowns, and run-time updates in accordance with variousembodiments.

FIG. 2 illustrates a CT scan image of a baggage item, and exampleseparated object images in a process in digital unpacking, and automaticoperator interactive resolution, in accordance with various embodiments.

FIG. 3 shows an example classification output of an automatic threatrecognition (ATR) process, as applied to a set of content item imagesoutput from a digital unpacking, using the example shown on FIG. 2 , indigital unpacking, and computer-directed operator interactive query inaccordance with various embodiments.

FIG. 4 shows an example operation in selective removal of unpacked itemimages from further consideration, using results of ATR classificationof unpacked content item images, such as generated by FIG. 2 , in aprocess in digital unpacking, removal, and operator interaction, inaccordance with various embodiments.

FIG. 5 shows an example of a system display of remaining images, and auser input field, in a graphical user interface (GUI) for operatorinteraction, in a process in digital unpacking, removal, and operatorinteraction, in accordance with various embodiments.

FIG. 6A shows an example further display on a system GUI for operatorinteraction for viewing a selected non-adjudicated item image in itsoriginal CT scan context, in accordance with various embodiments; FIG.6B shows another presentation of operator selection of anothernon-adjudicated object image in its original context, and FIG. 6C showsoperator selection of another non-adjudicated object image in itsoriginal context.

FIG. 7 shows operator GUI aspects in computer-based, automatic directedquery validation of operator input regarding an object image ofundetermined item type, in accordance with various embodiments.

FIG. 8 shows operator GUI aspects in computer-based, automatic directedquery validation of operator input regarding an object image ofundetermined item type, in accordance with various embodiments.

FIGS. 9A and 9B show operator display and GUI aspects in another exampleprocess in validating operator input regarding an object image ofundetermined item type, in accordance with various embodiments.

FIG. 10 shows an example configuration of a computer-generated secondaryscreening operator display for secondary screening to identify objectsof undetermined item-type, and unresolved threat class.

FIG. 11A, FIG. 11B, and FIG. 11C show a secondary screening displayconfiguration, and illustrates via snapshot sequence, using computerprompts and secondary screening responses to same, resolution inaccordance with various embodiments, as an aerosol can, of one examplenon-determined item-type, and unresolved threat class object image.

FIG. 12A and FIG. 12B show, as presented on an operator display,operations in a computer-directed query process for obtaining, inaccordance with various embodiments, via drill-down focusing, furtherinformation on an object corresponding to the object image resolvedusing operations illustrated in FIGS. 11A through 11C.

FIG. 13 shows a flow in a computer-based prompting an operator tocapture camera images of the object of which the type was determined,for this example, by operations shown on FIGS. 11A-11C, and aspects ofcomputer provision of guidance for operator entry of a textualdescription.

FIG. 14 shows aspects, on an example configuration of an operator GUI,and operations thereon computer-directed validating in accordance withone or more embodiments, including computer-presented prompts, and anexample text entry field for user entry, and a corresponding display forguiding such entry.

FIG. 15 shows an example flow of operations in a processes of CTscanning, digital unpacking, removal, operator interaction, andincremental ATR updating in accordance with one or more embodiments.

FIG. 16 is diagram of a feedback type iterative system updating 1600 inaccordance with various embodiments, illustrating embodiments' inherentconvergence to an all-knowing classifier through sequential updates inresponse to applying, upon every CT scanning, a digital unpacking,removing known type object images, following by computer directedquerying and verification of operator input.

FIG. 17 shows a flow 1700 in another embodiment, featuring a combinationof identifying, classifying, conferring between and among multiple ATRalgorithms, determining security actions, displaying, and collecting,providing inherent iterative updating though operation.

FIG. 18A shows a flow in a process including unpacking a CT scan imageinto separate object images. FIG. 18B show a flow in classifying theobject images into actionable threat categories, in accordance with oneor more embodiments.

FIG. 19A shows an example application to the same object images as usedin FIGS. 18A and 18B of a supplemental, different ATR in accordance withvarious embodiments; and FIG. 19B shows flow of example operations ofconferring among the multiple ATR classifications for conditionalre-classifying of initial single ATR determined threat categories, inaccordance with various embodiments.

FIG. 20A and FIG. 20B show an example determination of security outcome,utilizing preceding processes in digital unpacking, removal ofidentified object images, and computer-based directed query of operatorsfor resolving unknowns, in accordance with various embodiments.

FIG. 21 shows an example configuration of a display, e.g., in anoperator display, of an example unclassifiable items/objects, forcomputer-based, directed querying of operators, for analysis anddetermination.

FIG. 22 shows a flow of example operation in a collection of a groundtruth data on any item/object, send the ground truth date to secondarysearch, and associating the data to the CT image data file, orincorporation into subsequent algorithm updates in accordance withvarious embodiments.

FIG. 23 illustrates, in schematic form, a computing system on whichaspects of the present disclosure can be practiced.

DETAILED DESCRIPTION

In the drawings, like reference numerals designate identical orcorresponding parts throughout the several views. The drawings aregenerally not drawn to scale unless specified otherwise or illustratingschematic structures or flowcharts. As used herein, the words “a,” “an”and the like generally carry a meaning of “one or more,” unless statedotherwise.

One or more embodiments apply various computer-directed processes foroperator interaction and query, to efficiently resolve threatclassification of object images not classifiable by a current state ofthreat classifier system. In various embodiments, systems and methodscan include inherent updating with each resolution of an object imagethat, for example, due to not yet being classifiable by the system,invoked the computer-directed, query prompting of operators to obtaininformation for resolution. Features include, but are not limited to,automatic incremental updating, e.g., of classification algorithms or alibrary of objects, utilizing information obtained for the resolution.

One or more embodiments can provide, without limitation, variousfeatures for reduction of operator time and labor in resolvingunclassifiable object images. Such features include, but are not limitedto communicating to operators, in response to ATR or other classifiersidentifying one or more unclassifiable object images within a baggageitem CT scan image, an uncluttered, reduced content version of theimage. Systems and methods according to an embodiment can, for example,communicate to the operators a simplified version of the CT scan inwhich the unclassifiable object image is detected. Among such featuresis removing from the scan image sent to the operators, object images forwhich the threats category has already been determined, to an acceptableconfidence level. Example features can remove object images categorizedas benign objects. Such images can be distracting to the operators, andthere may no statistically justifiable basis for expending time andlabor inspecting what has already been inspected, and classified withacceptable confidence, as benign. Similarly, features can remove objectimages categorized as threat objects. Such images can be even moredistracting to the operators, and security actions and precautions havealready been instituted. Such embodiments can provide additionaladaptiveness, for example, by including context object images, toprovide operators with reference points to quickly find the one or moreindeterminable threat category object images.

Systems and methods according to an embodiment can also providepre-identified object boundaries, and can illuminate the objectboundaries, for example, on the operator's display screen. Features caninclude, in response to detecting a CT scan image having one or moreobject images of indeterminable threat category, communicating to theoperators display a context providing image. Features can also include,for example, communicating to the operators, with the context providingimage, an image of the one or more indeterminable threat category objectimages identified in the original scan of the entire baggage item.Further features can include a GUI feature through which the operatorcan elicit a brightening of the image boundary of the indeterminablethreat category object image. An example among such features, asdescribed in more detail in later sections, can include the operatorhovering the computer cursor over the object image, e.g., for more thana particular duration. The duration can be referred to as a hovertrigger duration. The computer in response, can highlight, e.g., changethe intensity or coloration or both, of the object images' boundary.

Description includes references to airport facilities and operations, asairports are an example environment in which various embodimentsaccording to this disclosure can be practiced. However, airportfacilities are not intended as any limitation as to environments inwhich embodiments in accordance with this disclosure can be practiced.

An example airport environment can include a baggage item CT scanner,e.g., which may be positioned, for example, in an access controlled areato prevent unauthorized contact or proximity to the equipment.Conveyance of baggage items to the CT scanner can be provided, forexample, by a belt conveyor, e.g., from a loading area. The CT scannermay include an internal conveyance positioning mechanism to maneuverbaggage items to a scan position. The environment can include a threatlevel classifier, for example, an automatic threat recognition (ATR)logic. The threat level classifier can be configured to receive CT scanimage, apply a boundary detection process to separate individual objectimages, and apply an object image threat level classifier, or aplurality of different object image threat level classifiers. The objectimage threat level classifiers may be applied in combination with one ormore object image type classifiers. Object image type classifiers caninclude one or more Q-class object image classifiers, “Q” being aninteger, which may have different values for different ones of suchclassifiers. In an embodiment, object image threat level classificationprocess may not include, at least in a discrete sense, object typeclassifiers. For example, in an embodiment, an object image threat levelclassifier or threat level categorization logic can be included than canclassify, for example, a shotput as a threat item, without generating anexplicit identifier of the object being a shotput. Such a classifier mayclassify based on a plurality of factors, some of which may be unrelatedto physical features of the objects. It will therefore be understoodthat systems and methods in accordance with one or more embodiments canbe implemented and practiced without including an object typeclassifier.

An example embodiment can include a switchable conveyor that, forexample, in correspondence to presence or absence of an alarm orequivalent from an object image threat level classifier, can route abaggage item the CT scanner to an appropriate one among a set ofdifferent potential destinations. One such destination can be loadingarea for loading the baggage item, for example, onto an aircraft.Another of such destinations can be a manual inspection area, e.g., forbaggage items detected, according to threshold confidence levels, asincluding certain kinds of threat items.

Features and benefits provided by various embodiments include providingto operators, through operator workstations, de-cluttered operatorsbaggage item scan images, from which all resolved threat level objectimages may have been digitally removed. Secondary benefits of thisexample feature can include, but are not limited to, reduced time andlabor load on the operators. In one or more embodiments features caninclude providing the operators with convenient guidance, via, e.g., byproviding context images by which CT scan images can be de-cluttered,while still retaining helpful context, e.g., adjacent, or overlappingobject images.

FIG. 1 illustrates an example system environment 100 for processes of CTscanning of containers, e.g., baggage items, featuring objectthreat-level classification and, in accordance with various embodiments,further includes, without limitation, processes of computer directedinteractive query resolution of unknowns, directed interactivevalidations, and run-time updates as described in more detail in latersections of this disclosure. The system 100 can include a CT scanapparatus 102, which can receive baggage items, such as the example BTM,from CT scan input conveyor 104. The system can include a switchableconveyor from the CT scan apparatus, to switchable deliver the baggageitem BTM to a cleared baggage area 103, a temporary holding area 105, ormanual inspection area 107 the latter also being referred to as a“secondary screening area” 107. Control and operation of the switchableconveyor system is described in more detail in the following paragraphs.The system 100 can include, in association with the CT scan apparatus102, a CT object image bounding logic 108, an object image threat levelclassification logic 110, and an object library 112. In an embodimentthe system 100 can include a baggage item routing logic 114, an operatorinteraction and object resolution logic 116. The system 100 can includean operator workstation 118, and a first conveyor switch 120, which canselect, under the control of the baggage item routing logic 114, amongfirst path 122 to the cleared baggage area 103, second path 124 to thetemporary holding area 105, and third path 126 to the secondaryscreening area 107. The system 100 can include a first conveyorapparatus 128 for implementing the first path 122, a second conveyorapparatus 130 for implementing the second path 124, and a third conveyorapparatus 132 for implementing the third path 126. The system 100includes a second conveyor switch 136, controlled by the baggage itemrouting logic 114, for selecting the routing of baggage items from thetemporary holding area 105, between a first exit path 138 to the clearedbaggage area 103, and a second exit path 140 to the secondary screeningarea 107. A first exit path conveyor 142 can implement the first exitpath 138 and a second exit path conveyor 144 can implement the secondexit path 140.

FIG. 2 illustrates a computer tomography (CT) scan image 202 of abaggage item, 204-1 and example set of content item images, as output bya digital unpacking of the CT scan image, in a process in digitalunpacking, removal, and operator interaction in accordance with variousembodiments. For purposes of description, the content item images willbe referred to as “object images.” Also, for purposes of thisdisclosure, the structure of the baggage item can be considered a“baggage item.”

Object images in the FIG. 2 example CT scan image comprise a firstobject image 204-1, which is a CT scan image of the baggage item, asecond object image 204-2, which is a CT scan image of a knife, and athird object image 204-3, which is a CT image of an improvised explosivedevice. Also included in the scan image 202 is a fourth object image204-4, and a fifth object image 204-5, and a sixth object image 204-6.The fourth object image 204-4 is a CT scan of a left shoe, and the fifthobject image 204-5 is a CT scan of a right shoe. The sixth object image204-6 is a CT scan image of a firearm, for purposes of description, theabove-described example first, second, third, fourth, fifth, and sixthobject images will be collectively referenced as “object images 204.” Inan embodiment, as represented by the individual graphic blocks at therightward region of the figure, a computer-implemented object boundingand separation process 206 has identified the respective boundaries ofand separated, i.e., unpacked the object images 204. It will beunderstood that “separation,” as used herein in the context of “boundingand separation” process 206, and recitation of “separate” and“separating” in reference to same, is a definitional separation. Thedefinitional separation can be but is not necessarily accompanied bycorresponding changes in physical storage locations in which the objectimages are stored.

For purposes of description, the object bounding and separation process206 will be assumed to not include classification of the respectiveunpacked object images 204, i.e., process 206 may not includeclassification of object type, e.g., firearm, shoe, aerosol can, and soforth, or of threat category.

FIG. 3 shows a result of applying a computer-implemented threat levelclassification process to the unpacked object images 204. The threatlevel classification process classifies the unpacked object images 204between being “benign” threat level objects, “threat” objects, and“unknown” threat level objects. The determined threat levelclassification is indicated by the form of the lines surrounds theunpacked object images 204. Object images 204 classified as threatobject images are indicated by a solid line border. The object imagesclassified as benign object images are indicated by a dashed lineborder, and object images classified as unknown threat level objectimages are indicated by a dotted line border. The legend is visible atthe upper region of FIG. 3 . As shown, the example object image threatlevel classifier classified the baggage item bag 204-1 and the left shoe204-4 as benign object images, classified the knife 204-2 and thefirearm 204-6 as threat object images. The example threat classifierclassified the right shoe 204-5, the IED 204-3, and the aerosol can204-7 as unknown threat level object images.

There can be various implementations of the threat classification logic.One example implementation can include classification as to object type,and separate classification as to threat level. In an embodiment, thelatter classifier can be implemented, for example, as a lookup table. Inan embodiment, the object image threat level classifier can classifybased, at least in part on the type classification. Inputs to such aclassifier can include, for example, type classifier results as to theobject type for the object images. In an embodiment, type classifierresults can include an affirmative result/not-affirmative resultclassification tag, where “affirmative” can mean the object imagematched a library object, to within a requisite matching threshold, andnot-affirmative can mean the object image did not match any libraryobject. Inputs to the object image threat level classifier can alsoinclude risk tags for the individual(s) corresponding to the baggageitem.

Referring to FIGS. 1 and 2 , example logic for the computer-implementedclassification process 200 can be provided by the object imageboundaries logic 108 and the object image threat level classificationlogic 110.

FIG. 4 shows example operations in selective removal of unpacked itemimages from further consideration, using results of ATR classificationas shown in FIG. 3 , of unpacked scan image object images, such asgenerated by FIG. 3 , in a process in digital unpacking, threatclassification, and computer directed query resolution of unknown threatlevel object images, in accordance with various embodiments. Operationsin the selective removal for computer directed query based resolution,can include removing 402 the benign object images, i.e., for thisexample, the baggage item object image 2041, and the left shoe objectimage 204-4. Operations can also include removing 406 the threat objectimages. In this example, these include the knife 204-2 and the firearm204-6. The remaining object images 204 are object images 204 ofunresolved threat level. In this example, unresolved are the right shoeobject image 204-5, the IED object image 204-3, and the aerosol objectimage 204-7. Operations in the FIG. 4 selective removals as visible onFIG. 4 can also include communication, of the unresolved threat levelobject images to the operator workstation 118, e.g., over a computernetwork serving the FIG. 1 computer resources. Features of theabove-described selective removals include, but are not limited to,providing to the operator workstation 118 a decluttered image of an itemby digitally removing, from the CT scan image data, the object imagesthat classify as threat object images and the object images thatclassify as benign object images.

Features can also include corresponding receiving 408 of suchdecluttered images by the operator workstation 118. For purposes ofdescription, such images, while displayed on the GUI of the operatorworkstation, e.g., the FIG. 1 operator workstation 118, will bealternatively referred as “subject images.”

FIG. 5 shows an example of an operator display 500 of subject images502, and proximal thereto, a computer-to-operator query presentationfield 504, and an operator response button 506. According to anembodiment, a computer resource, e.g., the FIG. 1 computer operatorinteraction, object resolution logic 116, can display in thecomputer-to-operator query presentation field 504 the following query,while the subject unknown threat level object images are presented onthe system display field: “Select All Possible Threat Items.” In anapplication, such items can be automatically annotated and referred forsecondary screening.

It will be understood that the scope of “possible,” in the context of“Select All Possible Threat Items,” may be environment-specific. In anembodiment, the operator display 500 can be implemented using an HTML,web-type configuration in which the user clicks on unresolved objectimages that the operator believes or judges as possible threat items,and can be designed such that the operator cursor on particular ones ofthe subject images 502 to select ones as being “possible threat items.”

In implementation of the operator display 500, an interaction mechanismfor selecting and not selecting subject images 502 as being “possiblethreat items” can be configured, for example, as click indications forthe subject images 502. In an embodiment, click indication can include afirst type of click indication on the display of the GUI for subjectimages 502 as being possible threat items, as opposed to a second typeof click indication for subject images 52 not seen as possible threatitems. The first type of click action can include, for examplehighlighting the display of the subject item, and the second type ofclick indication on the display of the GUI can include not highlightingthe display of the subject image. In an implementation, the buttonfunctionality of the operator response button 506 can be, for example, aclick field on which the operator can place the operator's cursor andclick, or can be a touchpoint on a touchscreen.

It will be understood, however, that “button” and “click,” in thecontext of the operator display 500 and its features and aspectsdescribed above, are logical functions. Physical implementation can be,but is not limited to conventional HTML interface configurations. Forexample, the operator response button 506 need not resemble a “button.”An implementation of the operator response button 506 can include, forexample, a right-click pull-down, which an operator can activate whilethe operator's cursor hovers above a particular one of the subjectimages 502. The pull-down can be configured to include, in its visibleoptions list, a select” or equivalent. In a further example of suchimplementation, configuration can include assigning a left click, withthe operator's cursor on the “select” option in the pull-down optionlist, as an operator action that designates the particular subject image502 as a threat item.

In the above-example implementation of the operator display 500, theexample computer-generated query appearing in the computer-to-operatorquery presentation field 504 was “Select All Possible Threat Items.” Inan example alternative implementation, the computer query could be“Click each Subject Image that You Can Identify a Threat Level, andEnter Your Response.” In such implementation, a right-click orequivalent may activate a pull-down list, having options that include“Threat Item” and “Benign Item.” In an embodiment, the options list canbe configured with more detailed granularity. For example, options mayinclude “Threat Item,” “Benign Item,” “Categorical Benign Item,” and“Categorical Threat Item.”

In an embodiment, the system, e.g., the FIG. 1 operator interaction,object resolution logic 116, can include a feature of communicating tothe operator GUI, with the set of subject images, a data for displayingpositions and arrangements of the subject images relative to positionsand arrangements of context object images that appeared in the CT imagewith the subject images. Features can also include, responsive to theoperator indicating a context request, displaying on the operator GUIthe particular subject image together with the corresponding the contextobject images. The feature can also include highlighting the subjectimage's object image, in this example the image of the IED 204-3, toavoid being lost in its context. The context request feature can beimplemented for example, by the operator hovering the cursor above theparticular subject image for a time meeting a threshold, e.g., a contextrequest threshold. Also, in one or more embodiments, a further featurecan include at least one of the context object images not being amongsubject images. In other words, the context object images can include atleast one object image that was logically removed, from the imageddisplayed on the operator's GUI.

An example implementation of the above-described feature will bedescribed in reference to FIGS. 6A, 6B, and 6C, in which FIG. 6A showsan example display on the operator GUI in which the operator hascommunicated the context request for the leftmost of the subject images.As described above, this may be performed, for example, by hovering thecursor over the leftmost subject image for a time exceeding the contextrequest threshold. As visible, the result is the IED image 204-3 beingvisible in its original context, while being highlighted to aboveconfusion. FIG. 6B shows operator GUI display resulting from operatorselection, e.g., by hovering the cursor over the middle of the threesubject images, and corresponding highlighting of the aerosol can 204-7,within its original context. FIG. 6C shows operator inputting a contextrequest, selection the rightmost subject image, the right shoe 204-5, asanother non-adjudicated object image in its original context.

FIG. 7 shows a configuration of an operator GUI display, after theoperator has identified two of the three subject images as “threatobjects,” as the computer requested of the operator via the computergenerated query presentation field 504. As shown, the border of the twoselected subject images has changed from the dotted line indication ofhaving an unknown threat level to the solid line border indication ofbeing a threat object. It will be understood that the above-describedvisible indication of object image threat level, i.e., a threat-levelindicative border style in accordance with the legend appearing in theupper region of FIG. 3 , is only an example. Alternative indicationsinclude, but are not limited to, a different array of border styles, andinclude annotation other than borders.

Referring to FIG. 7 , object image 204-5, in this example a right shoe,which can be referenced as subject image 204-5 while displayed on theoperator display, remains indicated as an object image of unknown threatlevel. In an embodiment, one or more identification processes may beapplied for such items, i.e., items that were unknown by the systemthreat classifiers, but not identified as a threat by the operator. Suchidentification processes can include the computer querying the operatorto identify the items. Implementation can include subject imagesremaining after the operator identification and designation described inreference to FIGS. 6A-6C, being automatically designated as subjectimages of unknown identity. For purposes of description, after anoperator identification and designation as described in reference toFIGS. 6A-6C, the now-designated subject images of benign threat can bereferred to as a “set of subject images of unknown identity.” If thereare no subject images of benign threat the set of subject images ofunknown identity is a null set.

FIG. 8 shows operator display, e.g., on a GUI display configuration 800,aspects in computer directed query validation of operator inputregarding an object image within the set of subject images of unknownidentity. In an embodiment, a computer directed query validation asillustrated in FIG. 8 can be initiated in response to the set of subjectimages of unknown identity not being a null set. The query can includedisplaying on the computer query field 504 of the operator display, as aquery image 802, a subject image among the set of subject images ofunknown identity. In the example visible on FIG. 8 , the query image issubject image 204-5, the right shoe of the example introduced inreference to FIG. 2 . In an embodiment, operations in the instantexample computer directed query validation can include displaying, e.g.,in the GUI display configuration 800, e.g., under or adjacent the queryimage 802, a text entry field 804. Operations can also include acomputer-generated query or prompt, e.g., in the computer querypresentation field 504, for the operator to enter an operator indicationin the text entry field 804 a type descriptor for the object indicatedby the query image. Responsive to an operator indication, e.g., via thesubmit button 804, that entry of the type descriptor is complete,operations include, based at least in part on the type descriptor thatthe operator entered in the text entry field 804, a computer resourcesuch as the FIG. 1 operator interaction, object resolution logic 116,assigning the operator entered type descriptor as a type of the subjectimage displayed as the query image.

Referring to FIG. 8 , in an embodiment, computer directed queryvalidation of operator input regarding an object image within the set ofsubject images of unknown identity can also include displaying, e.g., ina candidate type field 808 that is proximal to the text entry field 804,a sequentially updated candidate type, using a sequential updatecorresponding to operator sequential entry of text characters into thetext entry field 804. In an implementation, responsive to a user entryto the GUI indicating a candidate type is correct, e.g., by touching orclicking on the submit button 806, operations can include setting theoperator's entry of the type as being complete and as being thecandidate type.

In an embodiment, the system can provide a computer-directed validationof operator inputs for subject images of unknown type by querying theoperator with confirmatory questions using known items of the same typeas a baseline. An example can include displaying on the GUI of theoperator workstation a query image, an image of a library object, and atleast one other object image among the subject images. The image of thelibrary object can be from an object library, such as the FIG. 1 objectimage library 112. In an implementation, the library object can be of atype identical or similar to the operator-indicated type, e.g., the typeentered in the process described above in reference to FIG. 8 . In oneor more embodiments, operations in validating through computer-directedquerying with confirmatory questions, configured for confirmatoryresponse can include request for operator confirmatory response, on theGUI of the operator workstation, between affirmative and notaffirmative, as to matching between the library object and theoperator-indicated type of the query image.

FIGS. 9A and 9B show an example configuration for the above-describedquerying the operator with confirmatory questions using known items ofthe same type as a baseline.

FIG. 10 shows an example configuration of a computer-generated secondaryscreening operator display 1000 for secondary screening to identifyobjects images of undetermined item-type, and unresolved threat class.Referring to FIG. 10 , example operations can include displaying, e.g.,on a computer display viewable from the secondary inspection area 107, asecondary inspection display configuration 1000. The secondary displayconfiguration 1000 can include a guidance image 1002, which can be aconfiguration of the received CT scan image for the baggage item, e.g.,the FIG. 2 scan image 202. The secondary display configuration 1000 canalso include, e.g., in a geometrically organized configuration forinspection convenience, object images of the CT scan image, which areshown in FIG. 10 as a first secondary inspection object image 1004-1,second secondary inspection image 1004-2, . . . , and so forth, throughto and Nth secondary inspection image 1004-N, collectively “secondaryinspection images 1004.” In an embodiment, the secondary inspectionimages 1004 can include a type field 1006.

For purposes of a teaching example, the FIG. 10 visible secondaryinspection images 1004 are the unpacked object images 204 described inreference to FIG. 2 . In accordance with the type classification statusshown in FIG. 3 , secondary inspection images 1004 of known type,namely, threat object image types for this example, are the knife, whichis the first secondary inspection image 1004-1, and the firearm, whichis the fourth secondary inspection image 1004-4. FIG. 111A shows theobject fields 1006 of these two object images populated accordingly.

FIGS. 11A, 11B, and 11C show a secondary inspection displayconfiguration, and illustrate via snapshot sequence, using computerprompts and secondary inspector responses to same, resolution inaccordance with various embodiments, as an aerosol can, of one examplenon-determined item-type, and unresolved threat class object image.Referring to FIG. 11A, the secondary inspection display configuration isnumbered as item number 1100A, to distinguish from subsequent FIGS. 11Band 11C snapshots. The FIG. 11B secondary inspection displayconfiguration is numbered 1100B, and the FIG. 11C secondary inspectiondisplay configuration as 1100C.

The FIG. 11A secondary inspection display configuration 1100A includesthe content of the FIG. 10 secondary inspection display configuration1000, and can further include two or more GUI object image type buttons,or object image descriptor button, or both, and can include a benign ornon-threatening object selection button. The specific example visible inFIG. 11A includes, as GUI object image type selection buttons, IEDobject image type selection button 1102-1, an aerosol object image typeselection button 1102-3, and a firearm object image type selectionbutton 1102-5. The FIG. 11A example also includes GUI object physicalcharacteristics selection buttons, which include liquid gelcharacteristic selection button 1102-2 and sharp characteristic button1102-4, and includes non-threat object type selection button 1102-6.

Features of the FIG. 11A secondary inspection display configuration1100A include computer guidance, via a display of the CT scan image ofthe item container, e.g., of the baggage item, together with display ofthe unpacked object images, with known types being clearly marked by theimage type field 1106, and a visible assortment of GUI selection buttonsfrom which to quickly select. An implementation for entry of the objectitems can provide, for example, for the secondary inspector apply amouse button selection, e.g., by clicking on the image, of one of theobject images of unknown type, followed by clicking on one of the GUIselection buttons.

Referring to FIG. 11B, features of the secondary inspection displayconfiguration 1100B can include, but are not limited to, provision forthe secondary inspector to select one of the secondary inspection objectimages 1004, by positioning a mouse cursor 1104 the selected secondaryinspection object image, which for purposes of teaching example is shownas the third secondary inspection object image 1004-3. As shown in FIG.11B and as described above, the object is an aerosol can or container.In an embodiment, selection action by the secondary inspector can be amouse click, or a mouse hover time exceeding a trigger duration, whilethe cursor hovers over the desired secondary inspection image, i.e., thethird secondary inspection object image 1004-3. In one or moreembodiments, system response to the secondary inspector above-describedselection can include a system highlighting of the selected thirdsecondary inspection object image 1004-3. FIG. 11B represents thehighlighting by darker, thicker lines for the third secondary inspectionobject image 1004-3. In one or more embodiments, features can alsoinclude, without limitation, highlighting within the item container scanimage 1002 of the object image selected by the secondary inspector. Anexample operation of this feature is represented in FIG. 11B by darker,thicker lines for the outline 1106 of the aerosol can image as itappears in the CT scan image 1002 with its context object images.

Referring to FIG. 11C. features of the secondary inspection displayconfiguration 1100C can include, without limitation, provision for thesecondary inspector to select to select an object type or objectcharacteristic(s) to assign to the selected secondary inspection objectimages 1004. An example can include provision for the secondaryinspector to position the mouse cursor 1104, or equivalent, over theappropriate one of the GUI selection buttons 1102 and, e.g., withoutlimitation, either click or hover at said position for more than thehover trigger duration. click. In one or more embodiments, systemresponse to the above-described secondary inspector selection of anobject type or physical characteristic can include automatic,appropriate, populating of the object type field 1006. FIG. 11C showssuch action by darker, thicker lines for the aerosol GUI selectionbutton 1102-3, and by populating the object type field 1006 of the thirdsecondary inspection object image 1004-3 with “aerosol.”

FIGS. 12A and 12B show, as presented on an operator display, operationsin a computer-directed query process for obtaining, in accordance withvarious embodiments, via drill-down focusing, further information on anobject corresponding to the object image resolved using operationsillustrated in FIGS. 11A through 11C.

Referring to FIG. 12A, an implementation of the GUI displayconfiguration 1200A, can include displaying, under or adjacent queryimage 1202, a text entry field 1204A, and displaying, in the querypresentation field 504 of the GUI of the operator workstation, acomputer-generated query or prompt, asking entry, in the text entryfield 1204A, of a descriptor, or other operator indicator of an answerto the question “What Type of Aerosol Is this Item?” In animplementation the user, e.g., the secondary inspector, can click the“Submit” button 506 when the user believes e.g., is satisfied that theentry answers the computer query. In an embodiment, disincentives toentering information exceeding the scope of the computer query can beprovided. One such incentive can be a candidate descriptor field 1206A,in combination with a processor-executable instructions for aprogrammable computer to generate, display, and sequentially update acandidate descriptor corresponding to operator sequential entry of textcharacters into the text entry field 1204A. Implementations can includeprovision for user indication of the candidate descriptor beingappropriate.

FIG. 12B shows, via GUI display configuration 1200B, an implementationof a more detailed computer directed query that can include displaying,in the query presentation field 504, a computer-generated query orprompt, asking entry, in the text entry field 1204B, of a descriptor, orother operator indicator of an answer to the question “What Brand ofAerosol is this Item.” Operator entry of the response can be via textentry field 1204B and candidate descriptor field 1206B as describedabove in reference to FIG. 12A.

FIG. 13 shows a flow in a computer-based prompting an operator tocapture camera images of the object of which the type was determined,for this example, by operations shown on FIGS. 11A-11C, and aspects ofcomputer provision of guidance for operator entry of a textualdescription.

FIG. 14 shows aspects, on an example configuration of an operator GUI,and operations thereon computer-directed validating in accordance withone or more embodiments, including computer-presented prompts, and anexample text entry field for user entry, and a corresponding display forguiding such entry.

FIG. 15 shows a diagram of operation flow 1500 in a method for digitalunpacking of CT scan images, threat-level classification of known objectimage types and categoricals, computer-directed operator querying onunknowns, and incremental updating in accordance with one or moreembodiments. The flow 1500 includes receiving 1502 a CT scan image of,for example, a baggage item and proceeding to digital unpacking 1504 theCT scan image. Operations in the digital unpacking 1504 can includebounding all of the object images and may include generating respectiveobject image files that can include, for example, geometric data, CTobtained density data, and other information. Operations in the flow1500 can proceed from digital unpacking 1504 to classifying 1506 objectimages, into threat classes. In an embodiment, the classifying 1506 canclassify the threat levels into, for example, threat object class,categorical threat object class, benign object class, and categoricalbenign object class.

In an embodiment, operations in the flow 1500 can include receiving athreat category of an individual associated with the baggage item, or,for example, if the baggage item is associated with a travel group, athreat category of the group. Operations can include, in response,adjusting 1507A classification thresholds applied by the threat levelclassifying 1506, or adjusting 1507B granularity applied by theclassifying and in measurements obtained for the classification inputs,or both. As described above, various embodiments can include applying aplurality of classification algorithms in performing the classifying1506. A result of the adjusting 1507A of the classification thresholdsapplied by the threat level classifying 1506, and/or adjusting theconfiguration or selection of, or combination of algorithms applied inthe classifying 1506 can be an adjusted threat level classificationprocess. In an embodiment, the flow 1500 can include proceeding fromclassifying 1506 the threat levels of object images to a triggeringoperation such as determining 1508 whether there are any threat objectimages. In an embodiment, responsive to an affirmative determining 1508the flow 1500 can route 1510 the baggage item to an inspection area formanual inspection. FIGS. 1 and 15 , example operations in operation inrouting 1510 can include an alarm from the object threat levelclassification logic 110 to the baggage item routing logic 114. Thebaggage item routing logic 114. In response, can control the firstrouting switch 120 to send the baggage item on route 126 to thesecondary search area using, e.g., conveyance 132.

Referring to FIG. 15 , either concurrent with, or sequential withrouting 1510 the baggage item 1510 for manual inspection, the flow 1500can proceed to removing 1512 threat level resolved object images.Operations in the removing 1512, in other words, can remove objectimages for which no operator input is required, i.e., threat objectimages, and benign object images. Removed threat object images caninclude object images determined as threat objects based on object imagetype, as well as object images determined s categorical threats, e.g.,based on object physical parameters, with or without risk data on theindividual(s) associated with baggage item. Similarly, removed benignobject images can include objects images determined benign based onobject image type, or as categorically benign. The in order to send theoperator(s) only the non-resolved object images. It will be understoodthat “removing” in this context of “removing” 1512 can be a logicaloperation, e.g., “removing from further consideration by operators.”Above-described example operations in the removing 1512 can include,referring to FIG. 4 , removing 402 benign object images, and removing404 threat object images.

In an embodiment, flow 1500 operations after the removing 1512 candepend on whether any unresolved threat classification object imagesremain. If the answer is no, meaning all object images wereaffirmatively classified as to type, and hence as to risk level, ordetermined as having categorical risk level, operations can end. Ifthere were no threat object images identified by the classification 1506the baggage item will have traversed over FIG. 1 route 122 to thecleared baggage pick-up area. If threat object images were identified,subsequent routing of the baggage item can depend on the results of thesecondary inspection.

Referring to FIG. 15 , assuming the answer at 1514 is “yes,” meaningthere are unresolved threat level object images in the CT scan image,operations can proceed to computer directed query processes 1516,directed to resolving the threat level of all unresolved threat levelobject images. Above-described operations that can implement instancesof the computer-directed query process can include, referring to FIGS. 1and 4 , the operator interaction, object resolution logic 116communicating to the operator workstation 118, via the FIG. 4communicating 406 of unresolved object images, and the workstation 118receiving, e.g., via the FIG. 4 receiving 408, of such unresolved objectimages. Implementing operations can further include, for example,displaying of computer-directed queries as described in reference toFIG. 5 , and reception of operator responses to the queries, forexample, as described in reference to FIGS. 6A-6C, and FIG. 7 . Invarious embodiments, computer-directed queries can provide furtherguidance to operators, such as described above, e.g., in reference toFIG. 10 , and FIGS. 11A, 11B, and 11C.

In an embodiment, baggage items determined at 1514 as includingunresolved threat level object images can be temporarily conveyed to atemporary holding area such as the FIG. 1 temporary holding area 125.Referring to FIG. 1 , example operations can include, instructions fromthe from the baggage item routing Operations can include a command fromthe object threat level classification logic 110 to the baggage itemrouting logic 114 that, in turn, can control the first routing switch120 to send the baggage item on route 124 to the temporary holding area125. Routing of the baggage from the temporary holding area 125 candepend on the resolution obtained by the computer directed queryprocesses 1516. In an embodiment, responsive to resolution by thecomputer directed query processes 1516 that identifies no threatobjects, instructions can be provided to the second route switchinglogic 136, e.g., by operator interaction, object resolution logic 116,to convey the baggage item on route 140 to the cleared baggage itemareas. In like manner, in such embodiments, responsive to negativeresolution by the computer directed query processes 1516, e.g., onewhich identifies threat objects, logic such as operator interaction,object resolution logic 116 can instruct the second route switchinglogic 136 to convey the baggage item on route 142 to the secondaryinspection area.

In one or more embodiments, operations in the flow 1500 can includevalidating and confirming 1518 of operator entered object typeinformation. Example implementations of such confirming and validatingcan include, but are not limited to, processes and operations therein asdescribed above, e.g., in reference to FIGS. 8, 9A, 9B, 12A, and 12B.

Referring to FIG. 15 , there may be instances, in operations accordingto one or more embodiments, in which one or more object images in a CTscan image may appear irresolvable absent manual inspection. Suchinstances may occur, for example, in earlier training of the systems. Onthe other hand, such instances may not occur. Notwithstanding, suchinstances may be resolved by manual inspection, as shown by the dottedline from the validating and confirming 1518 to the manual inspecting1510.

Upon resolution from the validating and confirming 1518, the flow 1500can proceed to incremental system updating 1520, e.g., updating thelibrary of objects to add a new library object or by updating thealgorithm(s) applied by threat level classifying 1506, or both.

FIG. 16 shows a logic diagram of a feedback type iterative systemupdating 1600 in accordance with various embodiments, illustratingembodiments' inherent convergence to an all-knowing classifier throughsequential updates in response to applying, upon every CT scanning, adigital unpacking, removing known type object images, following bycomputer directed querying and verification of operator input. Referringto FIG. 16 , operations can include receiving a CT scan image 1602,digitally unpacking 1603 the CT scan image to feed the threat levelclassifying 1604, which classifies, in this example iteration, all ofthe unpacked object images 204 with the exception of the IED device204-3. To resolve the unknown threat level in this iteration, the flow1600 can proceed via removal 1606 to send the IED device 204-3 to thecomputer-directed querying 1608 via operator station GUI, and obtainingoperator input response. The computer-directed querying 1608 canidentify the object type of the IED device 204-3, and based thereon, candetermine the IED device 204-3 a threat device. The flow 1600 can theninclude feedback 1610 to update the threat level classifying 1604, e.g.,by adding the IED device 204-3 to the object library.

FIG. 17 shows a portion of a flow in another embodiment, featuring acombination of identifying, classifying, conferring between and amongmultiple ATR algorithms, determining security actions, displaying, andcollecting, providing inherent iterative updating though operation. TheFIG. 17 portion incudes receiving a CT scan image 1702 of a baggageitem, and digitally unpacking the CT scan image to obtain, as anassortment of content object images, a laptop computer 1704, a bag ofpotato chips 1706, a box cutter 1708, coins 1710, unknown objects 1712,shoes 1714, an unknown device 1716, and the baggage item 1718. It willbe assumed that the object image threat level classifier, described inmore detail in reference to FIGS. 18B and 19B, does not recognize thepotato chips 1706. In an embodiment, multiple independent ATR algorithmscan be configured to perform the unpacking or disassemblysimultaneously. Alternatively, or in combination, unpacking can beperformed by a central resource, e.g., in advance of CT data transitionto ATRs.

FIG. 18A shows the digital unpacking of FIG. 17 , in the context offeeding the FIG. 18B classifying 1800 of the object images intoactionable threat categories, in accordance with one or moreembodiments. FIG. 18B classifying 1800 can classify the threat levellaptop computer 1704, the shoes 1714, and the baggage item 1702 in knownclear class 1802, and the box cutter 1708 in known threat class 1804.The classifying 1800 also classifies the coins as known clearable class1806, the unknown device 1716 as known suspect class 1808, andclassifies the potato chips 1706 and unknown objects 1712 asunclassified threat level 1810.

In an embodiment, a method for adjudicating threat levels in CT scanimages can include receiving a CT scan image of an item container,including object images, and digitally unpacking the object images, toobtain a plurality of unpacked object images, as described above inreference to FIG. 17 and FIGS. 18A and 18B. In embodiment, the objectimage threat level classifier applied above can be a first threat levelclassifier, and operations can include, as described above, classifyingby the first threat level classifier the unpacked object images, resultsfrom the first threat level classifier including first classifier knownclear, first classifier known alarm, first classifier known clearable,first classifier known suspect, and first classifier unclassifiable. Inone or more embodiments, operations can also include classifying by asecond threat level classifier the unpacked object images, results fromthe second threat level classifier including second classifier knownclear, second classifier known alarm, second classifier known clearable,second classifier known suspect, and second classifier unclassifiable,and conferring between the results from the first level threat levelclassifier and the second level threat level classifier, and based on aresult of the conferring, re-assigning the threat levels.

FIG. 19A shows an example application to the same object images as usedin FIGS. 18A and 18B, of a supplemental, different ATR in accordancewith various embodiments. FIG. 19B shows flow of example operations ofconferring among the multiple ATR classifications for conditionalre-classifying of initial single ATR determined threat categories, inaccordance with various embodiments.

FIG. 20A and FIG. 20B show an example determination of security outcome,utilizing preceding processes in digital unpacking, removal ofidentified object images, and computer-based directed query of operatorsfor resolving unknowns, in accordance with various embodiments.

FIG. 21 shows an example configuration of a display, e.g., in anoperator display, of an example unclassifiable items/objects, forcomputer-based, directed querying of operators, for analysis anddetermination.

FIG. 22 shows a flow in example operations in a collection of a groundtruth data on an item or object, sending the ground truth date tosecondary search, and associating the data to the CT image data file, orincorporation into subsequent algorithm updates in accordance withvarious embodiments.

FIG. 23 illustrates, in schematic form, a computing system on whichaspects of the present disclosure can be practiced. The computing system2300 can include a hardware processor 2302 communicatively coupled to aninstruction memory 2304 and to a data memory 2306 by a bus 2308. Theinstruction memory 2404 can be configured to store, on at least anon-transitory computer readable medium as described in further detailbelow, executable program code 2309. The hardware processor 2302 mayinclude multiple hardware processors and/or multiple processor cores.The hardware processor 2302 may include hardware processors fromdifferent devices that can cooperate. The computing system 2300 systemmay execute one or more basic instructions included in the executableprogram code 2310.

Relationship Between Hardware Processor and Executable Program Code

The relationship between the executable program code 2309 and thehardware processor 2402 is structural; the executable program code 2309is provided to the hardware processor 2302 by imparting various voltagesat certain times across certain electrical connections, in accordancewith binary values in the executable program code 2309, to cause thehardware processor to perform some action, as now explained in moredetail.

A hardware processor 2302 may be thought of as a complex electricalcircuit that is configured to perform a predefined set of basicoperations in response to receiving a corresponding basic instructionselected from a predefined native instruction set of codes.

The predefined native instruction set of codes is specific to thehardware processor; the design of the processor defines the collectionof basic instructions to which the processor will respond, and thiscollection forms the predefined native instruction set of codes.

A basic instruction may be represented numerically as a series of binaryvalues, in which case it may be referred to as a machine code. Theseries of binary values may be represented electrically, as inputs tothe hardware processor, via electrical connections, using voltages thatrepresent either a binary zero or a binary one. These voltages areinterpreted as such by the hardware processor.

Executable program code may therefore be understood to be a set ofmachine codes selected from the predefined native instruction set ofcodes. A given set of machine codes may be understood, generally, toconstitute a module. A set of one or more modules may be understood toconstitute an application program or “app.” An app may interact with thehardware processor directly or indirectly via an operating system. Anapp may be part of an operating system.

Computer Program Product

A computer program product is an article of manufacture that has acomputer-readable medium with executable program code that is adapted toenable a processing system to perform various operations and actions.

A computer-readable medium may be transitory or non-transitory.

A transitory computer-readable medium may be thought of as a conduit bywhich executable program code may be provided to a computer system, ashort-term storage that may not use the data it holds other than to passit on.

The buffers of transmitters and receivers that briefly store onlyportions of executable program code when being downloaded over theInternet is one example of a transitory computer-readable medium. Acarrier signal or radio frequency signal, in transit, that conveysportions of executable program code over the air or through cabling suchas fiber-optic cabling provides another example of a transitorycomputer-readable medium. Transitory computer-readable media conveyparts of executable program code on the move, typically holding it longenough to just pass it on.

Non-transitory computer-readable media may be understood as a storagefor the executable program code. Whereas a transitory computer-readablemedium holds executable program code on the move, a non-transitorycomputer-readable medium is meant to hold executable program code atrest. Non-transitory computer-readable media may hold the software inits entirety, and for longer duration, compared to transitorycomputer-readable media that holds only a portion of the software andfor a relatively short time. The term, “non-transitory computer-readablemedium,” specifically excludes communication signals such as radiofrequency signals in transit.

The following forms of storage exemplify non-transitorycomputer-readable media: removable storage such as a universal serialbus (USB) disk, a USB stick, a flash disk, a flash drive, a thumb drive,an external solid-state storage device (SSD), a compact flash card, asecure digital (SD) card, a diskette, a tape, a compact disc, an opticaldisc; secondary storage such as an internal hard drive, an internal SSD,internal flash memory, internal non-volatile memory, internal dynamicrandom-access memory (DRAM), read-only memory (ROM), random-accessmemory (RAM), and the like; and the primary storage of a computersystem.

Different terms may be used to express the relationship betweenexecutable program code and non-transitory computer-readable media.Executable program code may be written on a disc, embodied in anapplication-specific integrated circuit, stored in a memory chip, orloaded in a cache memory, for example. Herein, the executable programcode may be said, generally, to be “in” or “on” a computer-readablemedia. Conversely, the computer-readable media may be said to store, toinclude, to hold, or to have the executable program code.

Creation of Executable Program Code

Software source code may be understood to be a human-readable,high-level representation of logical operations. Statements written inthe C programming language provide an example of software source code.

Software source code, while sometimes colloquially described as aprogram or as code, is different from executable program code. Softwaresource code may be processed, through compilation for example, to yieldexecutable program code. The process that yields the executable programcode varies with the hardware processor; software source code meant toyield executable program code to run on one hardware processor made byone manufacturer, for example, will be processed differently than foranother hardware processor made by another manufacturer.

The process of transforming software source code into executable programcode is known to those familiar with this technical field as compilationor interpretation and is not the subject of this application.

User Interface

A computer system may include a user interface controller under controlof the processing system that displays a user interface in accordancewith a user interface module, i.e., a set of machine codes stored in thememory and selected from the predefined native instruction set of codesof the hardware processor, adapted to operate with the user interfacecontroller to implement a user interface on a display device. Examplesof a display device include a television, a projector, a computerdisplay, a laptop display, a tablet display, a smartphone display, asmart television display, or the like.

The user interface may facilitate the collection of inputs from a user.The user interface may be graphical user interface with one or more userinterface objects such as display objects and user activatable objects.The user interface may also have a touch interface that detects inputwhen a user touches a display device.

A display object of a user interface may display information to theuser. A user activatable object may allow the user to take some action.A display object and a user activatable object may be separate,collocated, overlapping, or nested one within another. Examples ofdisplay objects include lines, borders, text, images, or the like.Examples of user activatable objects include menus, buttons, toolbars,input boxes, widgets, and the like.

Communications

The various networks are illustrated throughout the drawings anddescribed in other locations throughout this disclosure, can compriseany suitable type of network such as the Internet or a wide variety ofother types of networks and combinations thereof. For example, thenetwork may include a wide area network (WAN), a local area network(LAN), a wireless network, an intranet, the Internet, a combinationthereof, and so on. Further, although a single network is shown, anetwork can be configured to include multiple networks.

CONCLUSION

For any computer-implemented embodiment, “means plus function” elementswill use the term “means;” the terms “logic” and “module” have themeaning ascribed to them above and are not to be construed as genericmeans. An interpretation under 35 U.S.C. § 112(f) is desired only wherethis description and/or the claims use specific terminology historicallyrecognized to invoke the benefit of interpretation, such as “means,” andthe structure corresponding to a recited function, to include theequivalents thereof, as permitted to the fullest extent of the law andthis written description, may include the disclosure, the accompanyingclaims, and the drawings, as they would be understood by one of skill inthe art.

To the extent the subject matter has been described in language specificto structural features or methodological steps, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or steps described. Rather,the specific features and steps are disclosed as example forms ofimplementing the claimed subject matter. To the extent headings areused, they are provided for the convenience of the reader and are not tobe taken as limiting or restricting the systems, techniques, approaches,methods, or devices to those appearing in any section. Rather, theteachings and disclosures herein can be combined or rearranged withother portions of this disclosure and the knowledge of one of ordinaryskill in the art. It is intended that this disclosure encompass andinclude such variation. The indication of any elements or steps as“optional” does not indicate that all other or any other elements orsteps are mandatory. The claims define the invention and form part ofthe specification. Limitations from the written description are not tobe read into the claims.

Certain attributes, functions, steps of methods, or sub-steps of methodsdescribed herein may be associated with physical structures orcomponents, such as a module of a physical device that, inimplementations in accordance with this disclosure, make use ofinstructions (e.g., computer executable instructions) that may beembodied in hardware, such as an application specific integratedcircuit, or that may cause a computer (e.g., a general-purpose computer)executing the instructions to have defined characteristics. There may bea combination of hardware and software such as processor implementingfirmware, software, and so forth so as to function as a special purposecomputer with the ascribed characteristics. For example, in embodimentsa module may comprise a functional hardware unit (such as aself-contained hardware or software or a combination thereof) designedto interface the other components of a system such as through use of anapplication programming interface (API). In embodiments, a module isstructured to perform a function or set of functions, such as inaccordance with a described algorithm. This disclosure may usenomenclature that associates a component or module with a function,purpose, step, or sub-step to identify the corresponding structurewhich, in instances, includes hardware and/or software that function fora specific purpose. For any computer-implemented embodiment, “means plusfunction” elements will use the term “means;” the terms “logic” and“module” and the like have the meaning ascribed to them above, if any,and are not to be construed as means.

While certain implementations have been described, these implementationshave been presented by way of example only and are not intended to limitthe scope of this disclosure. The novel devices, systems and methodsdescribed herein may be embodied in a variety of other forms;furthermore, various omissions, substitutions, and changes in the formof the devices, systems and methods described herein may be made withoutdeparting from the spirit of this disclosure.

What is claimed:
 1. A method for adjudicating threat levels in computedtomography (CT) scan images, comprising: receiving a computed tomography(CT) scan image of an item container, including object images; digitallyunpacking the object images, to obtain a plurality of unpacked objectimages; classifying by a first threat level classifier the unpackedobject images, results from the first threat level classifier includingfirst classifier known clear, first classifier known alarm, firstclassifier known clearable, first classifier known suspect, and firstclassifier unclassifiable; classifying by a second threat levelclassifier the unpacked object images, results from the second threatlevel classifier including second classifier known clear, secondclassifier known alarm, second classifier known clearable, secondclassifier known suspect, and second classifier unclassifiable;conferring between the results from the first threat level classifierand the second threat level classifier; based on a result of theconferring, re-assigning threat levels; receiving, as an input to thefirst threat level classifier and the second threat level classifier, athreat classification of an individual associated with an itemcontainer; and adjusting the first threat level classifier and thesecond threat level classifier based at least in part on the threatclassification of the individual.
 2. The method of claim 1 foradjudicating threat levels in CT scan images, further comprising:applying an object bounding process to the CT scan image, configured todetect boundaries of the object images; and identifying the objectimages, based at least in part on the boundaries.
 3. The method of claim1 for adjudicating threat levels in CT scan images, further comprising:classifying the object images between threat object images, benignobject images, and unknown threat level object images; generating, as aresult, a set of threat object images, a set of benign object images,and a set of unknown threat level object images; displaying on a GUI ofan operator workstation a set of subject images comprising the set ofunknown threat level object images; and displaying, on the GUI of theoperator workstation a request that the operator enter a particularinformation for a subject image in the set of subject images.
 4. Themethod of claim 3 for adjudicating threat levels in CT scan images,further comprising: the request that the operator enter the particularinformation for the subject image in the set of subject images includesa request that the operator indicate, by first type of clickindications, the subject images that the operator assigns as threatobject images and, by second type of click indications, the subjectimages that the operator does not assign as threat object images.
 5. Themethod of claim 4 for adjudicating threat levels in CT scan images,further comprising: communicating to the GUI of the operatorworkstation, with the set of subject images, a data for displayingpositions and arrangements of the subject images relative to positionsand arrangements of context object images that appeared in the CT imagewith the subject images, at least one of the context object images notbeing among subject images; and responsive to the operator indicating acontext request, via the GUI of the operator workstation, for aparticular subject image, displaying on the GUI of the operatorworkstation the particular subject image together with the contextobject images.
 6. The method of claim 5 for adjudicating threat levelsof CT scan images, further comprising: detecting, as the operatorindicating the context request, the operator hovering a cursor over thesubject image, for a hover trigger duration.
 7. The method of claim 4for adjudicating threat levels in CT scan images, further comprising,the first type of click indications on the GUI of the operatorworkstation including a highlighting of the display of the subjectimage; and the second type of click indications on the GUI of theoperator workstation including not highlighting the display of thesubject image.
 8. The method of claim 4 for adjudicating threat levelsin CT scan images, further comprising: automatically designating, as aset of subject images of unknown type, subject images the operator doesnot assign as threat object images; and based on the set of subjectimages of unknown type not being a null set, performing a computerdirected query regarding object type, comprising: displaying on the GUIof the operator workstation, as a query image, a subject image among theset of unknown threat level object images, displaying on the GUI of theoperator workstation, proximal to the query image, a text entry fieldand, proximal the text entry field, a prompt for the operator to enterin the text entry field a type descriptor for the query image, and basedat least in part on an operator indication of completeness of the typedescriptor, assigning the type descriptor as a type of the subject imagedisplayed as the query image.
 9. The method of claim 8 for adjudicatingthreat levels in CT scan images, further comprising: displaying in acandidate type field that is proximal to the text entry field, asequentially updated candidate type, using a sequential updatecorresponding to operator sequential entry of text characters into thetext entry field; and responsive to a user entry to the GUI of theoperator workstation indicating a candidate type is correct, setting theoperator's entry of the type as being complete and as being thecandidate type.
 10. The method of claim 8 for adjudicating threat levelsin CT scan images, further comprising: associated with the assigning thetype descriptor as the type of the subject image displayed as the queryimage, removing from the set of subject images of unknown type thesubject image displayed as the query image; and conditional, afterremoving from the set of subject images of unknown type the subjectimage displayed as the query image, on the set of unknown threat levelobject images not being a null set: repeating the computer directedquery regarding object type, using a subject image from the set ofunknown threat level object images as another query image, and removingfrom the set of unknown threat level object images the subject imageused as the another query image.
 11. The method of claim 8 foradjudicating threat levels in CT scan images, further comprising:validating operator-indicated type of the query image, including:displaying on the GUI of the operator workstation, the query image, animage of a library object, and at least one other object image among thesubject images, the image of the library object being from an objectlibrary, the library object being of a type identical to or similar tothe operator-indicated type, displaying on the GUI of the operatorworkstation, with the subject image, the image of the library object,and at least one other object image from the CT scan image, a requestfor operator confirmatory response, on the GUI of the operatorworkstation, between affirmative and not affirmative, as to matchingbetween the library object and the operator-indicated type of the queryimage, and validating the operator-indicated type in response to theconfirmatory response being affirmatory.
 12. The method of claim 8 foradjudicating threat levels in CT scan images, further comprising:sending a guidance image, to a display of a secondary screening station,for a manual inspection of the item container, the guidance imageincluding the CT scan image of the item container, adjacent one or moreof the object images.
 13. The method of claim 1 for adjudicating threatlevels in CT scan images, further comprising: receiving as an input athreat classification of an individual associated with the itemcontainer; classifying the object images between resolved threat leveland unknown threat level object images including: adjusting an objectimage threat level classifier, to an adjusted threat levelclassification process, based at least in part on the threatclassification of the individual associated with the item container, theadjusted threat level classification process being configured toclassify object images between threat object images, benign objectimages, and unknown threat level object images; and applying theadjusted threat level classification process to the object images.
 14. Acomputer-based system for adjudicating threat levels in computedtomography (CT) scan images of item containers, comprising: a processor,and a tangible, non-volatile memory coupled to the processor and storingprocessor-executable instructions for the processor to: receive acomputed tomography (CT) scan image of an item container, includingobject images; digitally unpack the object images, to obtain a pluralityof unpacked object images; classify by a first threat level classifierthe unpacked object images, results from the first threat levelclassifier including first classifier known clear, first classifierknown alarm, first classifier known clearable, first classifier knownsuspect, and first classifier unclassifiable; classify by a secondthreat level classifier the unpacked object images, results from thesecond threat level classifier including second classifier known clear,second classifier known alarm, second classifier known clearable, secondclassifier known suspect, and second classifier unclassifiable; conferbetween the results from the first threat level classifier and thesecond threat level classifier; based on a result of the conferring,re-assign threat levels; classify the object images between threatobject images, benign object images, and unknown threat level objectimages; generate, as a result, a set of threat object images, a set ofbenign object images, and a set of unknown threat level object images;display on a GUI of an operator workstation a set of subject imagescomprising the set of unknown threat level object images; and display,on the GUI of the operator workstation a request that the operator entera particular information for a subject image in the set of subjectimages.
 15. The computer-based system of claim 14 for adjudicatingthreat levels in CT scan images of item containers, further comprisingthe processor-executable instructions further including instructions forthe processor to: perform an object bounding and separation process onimage data corresponding to the CT scan image to obtain the objectimages; apply an object image type classifier to the object images toclassify object images among a plurality of object image types; andconfigure an object image threat level classifier to classify betweenbeing a threat item image and being a benign item image, based at leastin part on an affirmative result from classification by the object imagetype classifier.
 16. The computer-based system of claim 14 foradjudicating threat levels in CT scan images of item containers, furthercomprising the processor-executable instructions further includinginstructions for the processor to: responsive to receiving operatorinput from an operator identifying a type for a subject image, perform acomputer-directed validation of the operator input, thecomputer-directed validation including instructions to: display to theoperator, on a GUI of an operator workstation, the subject image,together with an image of a library object from an object library, thelibrary object being of a type similar to the type identified by theoperator, and display to the operator, on the GUI of the operatorworkstation, a request for a confirmatory response as to matchingbetween the library object and the subject image.
 17. The computer-basedsystem of claim 14 for adjudicating threat levels in CT scan images ofitem containers, further comprising the processor-executableinstructions further including instructions for the processor to:perform a computer directed querying of an operator, for particularinformation regarding one or more subject images; and responsive to asuccessful outcome of the computer directed querying of the operator, toperform a resolving of threat level classification between a threat itemimage and a benign item image, and an updating of an object image threatlevel classifier.
 18. The computer-based system of claim 14 foradjudicating threat levels in CT scan images of item containers, furthercomprising the processor-executable instructions further includinginstructions for the processor to: receive, as an input to the firstthreat level classifier and the second threat level classifier, a threatclassification of an individual associated with an item container; andadjust the first threat level classifier and the second threat levelclassifier based at least in part on the threat classification of theindividual.
 19. The computer-based system of claim 14 for adjudicatingthreat levels in CT scan images of item containers, further comprisingthe processor-executable instructions further including instructions forthe processor to: generate a decluttered image of an item by digitallyremoving, from image data, the object images that classify as threatobject images and the object images that classify as benign objectimages.
 20. The computer-based system of claim 14 for adjudicatingthreat levels in CT scan images of item containers, wherein the objectimage type classifier includes one or more Q-class object imageclassifiers, “Q” being an integer; and wherein the object image threatlevel classifier is based on a lookup table.